Debugger for Multi-level Hybrid Parallel Programs on Heterogeneous Accelerator Cluster Architectures – Survey and Challenges

The need to debug hybrid parallel programs on heterogeneous accelerator clusters opens a new set of challenges for concurrently managing the processes and threads at node and accelerator levels. Currently, there exist open source debuggers for traditional HPC clusters, which support debugging of multi-node parallel programs. At present, debugging at the accelerator level is handled through language-specific debuggers provided by the device manufacturers. It is desired that a standards based debugger for hybrid parallel programs should support multiplelanguages, multiple devices, ease of use, scalability, and portability. However, currently there is no open-source standardized debugger for hybrid parallel programs on heterogeneous multi-accelerator clusters. The authors in this paper survey the existing debugger solutions for multi-level hybrid parallel programs. The authors in this paper bring out the challenges involved in developing a standardized open source debugger based on GDB, for heterogeneous accelerator clusters and present the features of a debugger on heterogeneous multiaccelerator clusters.